Deep indexed active learning for matching heterogeneous entity representations

Given two large lists of records, the task in entity resolution (ER) is to find pairs from Cartesian product that correspond same real world entity. Typically, passive learning methods on such tasks require amounts labeled data yield useful models. Active Learning a promising approach for ER low resource settings. However, search space, informative samples user label, grows quadratically instance-pair making active hard scale. Previous works, this setting, rely hand-crafted predicates, pre-trained language model embeddings, or rule prune away unlikely product. This blocking step can miss out important regions space leading recall. We propose DIAL, scalable jointly learns embeddings maximize recall and accuracy matching blocked pairs. DIAL uses an Index-By-Committee framework, where each committee member representations based powerful transformer highlight surprising differences between matcher blocker creation training objective used train their parameters. Experiments five benchmark datasets multilingual record dataset show effectiveness our terms precision, running time. Code available at https://github.com/ArjitJ/DIAL